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Patent 199 41 742.3 filed September 02, 1999 by Hoffmann et al. entitled CIRCUIT FOR 
GENERATING IMAGE DATA FOR A PC AND APPROPRIATE DATA TRANSFER 
METHOD.- 



Please cancel claims 1-18. 

Please add the following new claims. 
19. (New) Circuit for generating image data for real-time representation on a computer, with 
a data input for receiving raw image data of an image sensor, a buffer arranged to receive the 
inputted raw image data, a serial data output, and control logic, characterized in that the 
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control logic is provided for combining the buffered raw image data into data packets with 
protocol elements of the lowest Ethernet protocol level, and the serial data output is arranged 
to output the data packets. 

20. (New) Circuit according to claim 1 9, characterized in that the protocol elements in the 
data packets are only of the lowest Ethernet protocol level. 

21 . (New) Circuit according to claim 19, characterized in that the buffer has a capacity for 
the raw image data of at least one image line. 

22. (New) Circuit according to claim 21 , characterized in that the buffer has a capacity for at 
least two image lines. 

23. (New) Circuit according to claim 22, characterized in that the buffer is a dual-port 
random access memory. 

24. (New) Circuit according to claim 22, characterized in that a bank change-over switch 
between at least two memory blocks is provided, wherein each memory block has the 
capacity for at least one image line, and wherein the raw image data of sequential image lines 
are read into the respective other memory block and read-out in a pull-push manner. 

25. (New) Circuit according to claim 19, characterized in that the data input is a parallel input 
of a pre-determinable bit width, wherein the read-out or respectively the output frequency for 
the image data combined into data packets is greater than the product of a read-in frequency 
multiplied by the bit width of the data input. 

26. (New) Circuit according to claim 24, characterized in that the data input of the circuit 
has a 16 bit width. 
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27. (New) Circuit according to claim 19, characterized in that the data output is an optical 
output. 

28. (New) Circuit according to claim 19, further comprising a computer that is equipped 
with an Ethernet network card, characterized in that the PC is configured for receiving image 
data from the circuit, with driver software for the Ethernet network card that uses only 
protocol data from the lowest Ethernet protocol level as protocol data, and interprets all 
remaining data in the data packets received as image data. 

29. (New) Circuit according to claim 19, characterized in that the Ethernet network card is 
provided with an optical input. 

30. (New) Method for transmitting image data in real-time from a camera to a computer, 
wherein the raw image data are buffered and combined into Ethernet data packets, 
characterized in that only the lowest protocol level is used for combining the raw image data 
into Ethernet data packets, and the data are provided to the input of an Ethernet network card 
of the computer, wherein the Ethernet network card is operated such that only the data of the 
lowest protocol level is used as protocol data, and all remaining data are interpreted as image 
data. 

3 1 . (New) Method according to claim 30, characterized in that the raw image data is 
buffered line-by-line and combined line-by-line into Ethernet data packets and provided 
serially to the input of the Ethernet network card. 

32. (New) Method according to claim 31, characterized in that the data are read line-by-line 
into different memory blocks, wherein the data are subsequently read-out again in the same 
sequence from the memory blocks. 

33. (New) Method according to claim 31, characterized in that two memory blocks are 
provided, which raw image data are alternately read into and read from in a pull-push manner. 
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34. (New) Method according to claim 32, characterized in that reading out of the data of one 
image line takes place in a sequence pre-determinable by a control logic and independent of 
the reading-in of the image data of this image line. 

35. (New) Method according to claim 32, characterized in that at the same time as the 
reading-in of the image data in an image line, reading-in of the associated protocol data of the 
lowest Ethernet level takes place. 

36. (New) Method according to claim 30, characterized in that the buffered image data is 
only combined into Ethernet data packets and expanded with appropriate protocol data when 
a read-out takes place. 

37. (New) Method according to claim 30, characterized in that the data of an image line are 
respectively combined into a whole number of Ethernet data packets such that none of the 
data packets contains data from different image lines. 
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REMARKS 

Applicants respectfully request examination of the claims on the merits in this 
application. Applicants submit that the above claims represent allowable subject matter. The 
Examiner is encouraged to contact the undersigned to resolve efficiently any formal matters 
or to discuss any aspects of the application or of this response. Otherwise, early notification 
of allowable subject matter is respectfully solicited. 



One Dayton Centre 
One South Main Street, Suite 500 
Dayton, Ohio 45402-2023 
Telephone: (937) 223-2050 
Facsimile: (937) 223-0724 



Respectfully submitted, 

KILL WORTH, GOT^MAN, HAG^ 
SCHAEFF, L. 
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Circuit for generating image data for a PC and appropriate method for transfer of data 

The present invention relates to a circuit for generating image data in real-time 
representation on a PC, with a data input for receiving raw pixel data that is generally 
generated by a semi-conducting, light-sensitive image detection device. 

5 The present invention also relates to a personal computer (PC) that has an Ethernet card 
that is configured for receiving and representing appropriate image data. 

Lastly, the present invention also relates to a method for transmitting and representing on 
a PC, in real-time, image data that is generated by an image detection device. 

In principle, so-called matrix cameras or cell cameras, and respectively matrix sensors and 
10 cell sensors, can be considered as image detection devices. Because of the limited data 
capacity and data processing speed with a PC, image representation on a PC in real-time 
is a problem that until now has been overcome only using relatively expensive and 
complex additional components. 

In representing images that have been generated using a video camera, very large 
is quantities of data are soon accumulated. Even a conventional CCD sensor with 400,000 
image points generates a data rate of approximately 14 to 15 Mbytes/s. Rapid line 
sensors that have, for example 7,000 image points per line generate quantities of data in 
the region of 100 Mbytes/s and more. The problem of rapidly processing such quantities 
of data is conventionally solved by so-called frame-grabber circuits that are additional 
20 components for a PC and are relatively expensive. 

With respect to this prior art, the object of the present invention is to provide an 
appropriate circuit, an appropriately configured PC and an appropriate method for 
transmitting the data from an image sensor to a PC, in which the complexity of 
representing image data on the PC is considerably reduced and consequently the costs for 
25 producing real-time representation are also considerably decreased. 

With respect to the circuit described in the introduction, this object is solved in that the 
circuit has a buffer, control logic and a data output, wherein the control logic is provided 



for assembling the raw image data stored in the buffer into data packets and for 
generating protocol data for the data packets, wherein the protocol data are limited to the 
lowest level of an Ethernet protocol, 

"Raw image data" is understood here to be the already digitised values of the original 
analog pixel values that were generated by the image sensors. 

Such a circuit, with a buffer and comparatively simple control logic that simply assembles 
the raw image data into data packets and must provide it with at least one part of the 
protocol data from the lowest Ethernet protocol level is comparatively simple and 
inexpensive to manufacture. The data available at the corresponding serial data output of 
such a circuit can be directly supplied to the input of an Ethernet network card that is in 
any case part of the standard equipment of very many PCs. The PC then simply needs 
appropriately configured driver software for operating the Ethernet network card for 
appropriate conversion of the image data so that only the protocol data of the lowest 
Ethernet level (MAC layer) are used, and all other data are interpreted as image data. The 
Ethernet is capable of processing data rates o f 125 Mbytes/s (= 1 Gbits/s). It is thus also 
possible to receive and process appropriate image data at a data rate of up to 125 
Mbytes/s. In summary, the essence of the invention is in the exploitation of the 
possibilities of Ethernet network cards, wherein these are nevertheless not used in their 
normal operating mode in which the protocol data of several protocol levels are evaluated 
and the data processed accordingly, but instead used in a way for which it was not 
intended, in that only the data format limited to the lowest protocol level is generated, and 
wherein the driver software has then also to be configured appropriately for the Ethernet 
network card in order to be able to process the data without protocol data from the 
higher protocol levels of the Ethernet. The circuit according to the invention can either be 
integrated into a camera, it can be integrated into a PC input in front of an Ethernet card, 
or be provided as a separate, interconnectable component or interface element. When the 
circuit is provided as a separate component or input circuit on a PC, optionally several 
cameras with appropriately low data rates can also be connected at the same time to the 
input of the circuit according to the invention. For example, up to 8 conventional CCD 
matrix cameras could be set out parallel to the input of the circuit according to the 
invention, as together they generate a data rate of less than 120 Mbytes/s. 



It is advantageous when, with the circuit according to the invention, the buffer has a 
capacity for the raw image data of at least one scanning line. 

It is advantageous when the buffer has a capacity for at least two scanning lines. In such a 
case, the raw image data of a scanning line can firstly be completely written into an 
appropriate memory block before it is read from this area and output as data packets. 
While this reading and data output is taking place, the next scanning line can in the 
meantime be written into the remaining memory block, and while this latter memory block 
is subsequently being read from, the former memory block can be written to again. 
Naturally, it would also be possible to use further memory blocks, however when the 
read-out speed is greater than the data rate at which the image data arrive, and 
respectively have to be written into the memory, two memory blocks are sufficient for one 
scanning line each. In this way costs for otherwise expensive memory space is also saved, 
particularly as these must be memories into which data can be written and read out at an 
appropriate speed. An embodiment of the invention is particularly preferred in which a 
so-called dual-port RAM is used as the buffer, that is to say a memory with selectable 
access and a dual connection, namely one connection for writing and respectively for 
memory, and on connection for read-out from the memory, wherein these two 
connections can be operated at different data rates or respectively pulse frequencies. 

Such a dual-port RAM preferably has two memory blocks, also known as "banks" 
wherein a bank change-over switch switches back and forth alternately between the two 
banks, wherein at a given time one bank is written into and the other is read from, and the 
writing and read-out of the two banks takes place continuously in a push-pull manner. 

Writing and reading of respective complete lines has the advantage that the raw image 
data can, when required, be written in a sequence into the line memory concerned, that is 
to say the bank, other than that in which it will possibly be read out subsequently. With 
many line sensors it is usual, for example, for the data for the individual image pixels to be 
sensed from the centre of the sensor towards the two sides, wherein in addition the line 
sensor can also be divided into several sectors in which this takes place separately. This 
means that the sequence of image data from the line sensor does not correspond to the 
line pattern from one end to the other, wherein, however, by reading out and with a 



known sequence for the writing procedure, the data can be arranged and output serially 
from one line end to the other line end, as soon as the line is completely memorised in the 
appropriate memory block, that is to say the bank. 

Furthermore, data writing can also be done parallel, for example with a bit width of 8, 1 6 
or 32 bits, whereas read-out takes place serially, that is to say the possibly parallel read- 
out procedure follows parallel/serial data conversion. Clearly, the read-out or respectively 
the output frequency must be greater than the writing frequency by at least the bit width 
factor. 

Where there is a higher data transmission speed, which is necessary for the circuit 
according to the invention, it has proved advantageous when the data output is an optical 
output, and when correspondingly the associated PC input, in concrete terms the input for 
an Ethernet network card, is an optical data input. 

Apart from the feet that it must be equipped with an Ethernet network card, the 
corresponding PC need only be have relatively simple driver software for receiving the 
image data, wherein the driver software simply ensures that of the image data, combined 
into data packets, being received, only the protocol data, or at least a part of the protocol 
data, of the lowest Ethernet protocol level are used (MAC layer), while all other data 
(which in the case of a conventional Ethernet data packet contain further higher levels of 
protocol data) are exclusively interpreted as image data. 

In this way it is possible to maintain the high image data transmission speed. As described 
hereinabove, the PC should also have an optical input in order to be able to receive the 
data, possibly output via an optical output by the circuit. 

With respect to the method described hereinabove, the object of the invention is solved in 
that the raw image data is buffered and combined into data packets in the Ethernet 
format, wherein, nevertheless, only Ethernet protocol data of the lowest protocol level are 
used, and these data combined into packets are then output serially and subsequently 
interpreted via the driver software of an appropriate Ethernet network card as pure image 
data that simply have additional data of the lowest Ethernet protocol level. 

Further advantages, features and possibilities for application of the present invention will 



be evident from the following description of a preferred embodiment and the attached 
drawings. There is shown, in: 

Figure 1 a block diagram that shows the principles of the circuit according to the 
invention and the connection thereof to a camera on the one hand, and to several PCs on 
the other, and 

Figure 2 a block diagram of a special embodiment of the invention, shown in rather more 
concrete terms. 

Figure 1 shows on the left an optical camera system 4 and located behind it a line sensor 5 
of a line camera. The line is divided internally in the line sensor into four line blocks, 
wherein the individual image data of these blocks is digitised and written, via multiplexers 

6 into a dual-port RAM 1 . The dual-port RAM 1 is for its part divided into two blocks 
la, lb, which by means of a change-over switch 1 1, shown here only schematically, and 
which can equally well be in the form of software control, alternately makes free one of 
the two blocks la, lb of the dual-port RAM 1 for writing in data from the multiplexers 6 
or respectively from the line sensor 5, while the respective remaining memory block lb or 
respectively la is made free for read out. The writing and reading procedures into and 
from the dual-port RAM 1 are controlled by control logic 3. The memory blocks, also 
called "banks" are dimensioned such that they can respectively receive the data of a 
complete image line from the line sensor 5. Regardless of the sequence, it is assumed that 
firstly the digitised image data of a complete image line will be written into the memory 
block, that is to say the bank 1 a. After a complete line of an image has been written into 
this memory block la, a switch-over is made to the memory block lb using the switching 
device 1 1 , and the next following line from the line sensor 5 is written in digitised form 
into the memory block, that is to say the bank lb. During this time, the data previously 
stored in the bank la is read out and arranged serially via the optical wave-guide interface 

7 in a pre-determinable sequence, that is to say parallel/serial conversion takes place. 
Moreover, the data are converted from an electrical into an optical form, and transmitted 
via the optical output 2 to a passive distributor 9. From this passive distributor, the data 
go to an Ethernet network card 13 of a PC. In the present instance, four different PCs 10 
are shown that are loaded with data from, for example, four sequential entities, one after 



another by the distributor 9. This means that the passive distributor first supplies all the 
sequential image lines of a given image or of a given series of images to a first PC 1 0, the 
next image or the next series of images is given to the next PC 20, and so on, so where 
necessary parallel image processing of sequential images can take place at the same time 
on several PCs. This is of interest, for example, when an appropriate camera is used in an 
optical sensing and sorting system, for example a parcel sorting installation. The line 
camera 4, 5 then possibly records different elements or portions of an entity that are 
respectively evaluated separately on a different PC. It is also possible for the camera to 
sense several different entities rapidly one after another, wherein the evaluation of the 
images of different entities takes place on different PCs. 

In addition to control of the writing and reading procedures in the dual-port RAM and in 
addition to switching over the switching device 1 1 , the control logic also fulfils other 
tasks such as, for example, control of the image line recording and read-out. During the 
read-out, using the interface 7 the line data can, for example, be divided into several data 
packets in accordance with the Ethernet standard, and provided additionally with 
appropriate frame data that are then given serially via the output 2 to the respective 
computers. An additional data input is shown at 8, via which the PCs 10 connected for 
image evaluation possibly deliver check-back or feedback signals that make suitable 
adaptation of the control logic 3 possible. 

The circuit according to the invention, which essentially only consists of the dual-port 
RAM 1, the control logic 3, and possibly also the interface 7, is relatively simply 
constructed and can therefore cope well with the large data rates. Moreover this circuit is 
substantially cheaper to manufacture than a frame-grabber card, and additionally such a 
circuit needs only to be provided once for several PCs, as shown in the embodiment. 

Figure 2 shows a somewhat modified embodiment of the circuit according to the 
invention, again in detail. A camera interface is labelled 12, which can, for example, 
satisfy the function of the components in Figure 1 represented by the multiplexers 6 and 
the series-connected units for analog/digital conversion of the image data received. 

A dual-port RAM 1 with two memory banks is also provided in this case, wherein each of 
the memory banks la, lb has an additional sub-sector lc and respectively Id, in which the 



frame data is stored according to the Ethernet protocol in the lowest protocol level, 
together with the data stored in areas la and respectively lb. In this case the data from 
memory banks la and respectively lb can be read directly together with the frame data 
from areas lc and Id, without the frame data having to be added in addition. All read and 
write procedures are controlled by the control logic 3 'substantially in the same manner as 
was described for the previous embodiment according to Figure 1 . The image data 
previously provided with the protocol frames from the lowest Ethernet protocol level are 
passed on via the control device 3' to the interface 7, in that a parallel/serial conversion 
takes place, and lastly the electrical data are converted into optical signals and transmitted 
in the optical wave guide controller 2 serving as an output. 

The control logic V is additionally provided with connecting lines to an input/output 
interface 14 and a further control that affects the camera interface 12 in order to make 
possible feedback control of the focussing of the optical system and/or image sensing, 
possibly as a reaction to available image results. 
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Claims 



1 . Circuit for generating image data for real-time representation on a PC, with a data 
input for receiving raw pixel data of an image sensor, characterised in that the 

5 circuit has a buffer (1), a serial data output (2) and control logic (3), wherein the 

control logic (3) is provided for combining the buffered raw image data into data 
packets with protocol elements of the lowest Ethernet protocol level (MAC 
layer). 

2. Circuit according to claim 1, characterised in that the buffer has a capacity for the 
io raw image data of at least one image line. 

3. Circuit according to claim 2, characterised in that the buffer has a capacity for at 
least two image lines. 

4. Circuit according to claim 3, characterised in that the buffer is a dual-port RAM. 

5. Circuit according to claim 3 or 4, characterised in that a bank change-over switch 
15 between at least two memory blocks (banks) is provided, wherein each memory 

block (bank) has the capacity for at least one image line, and wherein the raw 
image data of sequential image lines are read into the respective other memory 
block and read-out in a pull-push manner. 

6. Circuit according to one of claims 1 to 5, characterised in that the data input is a 
20 parallel input of a pre-determinable bit width, wherein the read-out or respectively 

the output frequency for the image data combined into data packets is greater than 
the product of the read-in frequency multiplied by the bit width of the data input. 

7. Circuit according to claim 6, characterised in that the data input of the circuit has 
a 16 bit width. 



25 8. 



Circuit according to one of claims 1 to 7, characterised in that the data output is 
an optical output. 



9. Personal computer (PC) that is equipped with an Ethernet network card, 
characterised in that the PC is configured for receiving image data from the circuit 
according to one of claims 1 to 8, with driver software for the Ethernet network 
card that uses only protocol data from the lowest Ethernet protocol level as 
protocol data, and interprets all remaining data in the data packets received as 
image data. 

10. PC according to claim 9, characterised in that the Ethernet network card is 
provided with an optical input. 

1 1 . Method for transmitting image data in real-time from a camera to a PC, 
characterised in that the raw image data are buffered, combined into Ethernet data 
packets in which only the lowest protocol level is used, and are provided to the 
input of an Ethernet network card of the PC, wherein the Ethernet network card 
is operated such that only the data of the lowest protocol level (MAC layer) is 
used as protocol data, and all remaining data is interpreted as image data. 

12. Method according to claim 1 1, characterised in that the raw image data is 
buffered line-by-line and combined line-by-line into Ethernet data packets and 
provided serially to the input of the Ethernet network card. 

13. Method according to claim 12, characterised in that the data are read line-by-line 
into different memory blocks, wherein the data are subsequently read-out again in 
the same sequence from the memory blocks. 

^14. Method according to claim 13, characterised in that two memory blocks are 

provided, which raw image data are alternately read into and read from in the pull- 
push manner. 

15. Method according to claim 13 or 14, characterised in that reading out of the data 
of one image line takes place in a sequence pre-determinable by the control logic 
and independent of the reading-in of the image data of this image line. 



16. Method according to one of claims 11 tol 5, characterised in that at the same time 
as the reading-in of the image data in an image line, reading-in of the associated 
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protocol data of the lower Ethernet level takes place. 

17. Method according to one of claims 1 1 to 15, characterised in that the buffered 
image data is only combined into Ethernet data packets and expanded with 
appropriate protocol data when read-out takes place. 

1 8. Method according to one of claims 1 1 to 1 7, characterised in that the data of an 
image line are respectively combined into a whole number of "Ethernet" packets 
such that none of the data packets contains data from different image lines. 
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